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DETAILED ACTION 

Response to Arguments 

1 . The Request for Continued Examination filed 2/05/2008 have been entered and 
made of record. 

2. The Applicant has canceled claim(s) 4 and 16. 

3. The application has pending claim(s) 1-3, 5-15, and 17-26. 

4. In response to the Request for Continued Examination filed on 2/05/2008: 
The "Objections to the specification" have been entered and therefore the 

Examiner withdraws the objections to the specification. 

The "Objections to the claims" have been entered and therefore the Examiner 
withdraws the objections to the claims. 

The "Claim rejections under 35 U.S.C. 112, second paragraph" have been 
entered and therefore the Examiner withdraws the rejections under 35 U.S.C. 112, 
second paragraph. 

The "Claim rejections under 35 U.S.C. 101" have been entered and therefore the 
Examiner withdraws the rejections under 35 U.S.C. 101. 

Regarding Claim 26 [lines 23-25], the Applicant has inadvertently forgotten to 
underline [underline to indicate current amended claim limitations] the amended 
limitation of "wherein the step of extracting a face image includes a step of specifying a 
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size of the face image to be extracted, based on a result of recognition by the distance 
recognition step". The Examiner has recognized this minor informality for the record. 

5. Applicant's arguments with respect to claim(s) 1-3, 5-15, and 17-26 have been 
considered but are moot in view of the new ground(s) of rejection because of the 
Request for Continued Examination (RCE). 

6. Applicant's arguments, see page 15 [starting at paragraph "The Examiner 
acknowledges ...] through page 16 [ending of paragraph "Thus, it would be clear to one 
skilled ...], filed 2/05/2008, with respect to the rejection(s) of claim(s) 1-3, 5-15, and 17- 
26 under 35 U.S.C 103(a) have been fully considered and are persuasive. Therefore, 
the rejection has been withdrawn. However, upon further consideration, a new 
ground(s) of rejection is made in view of the newly found prior art reference Kojima et al 
(US 5,638,1 36). Kojima discloses the amended limitations of a distance recognition 
part / calculate object distance L for recognizing a distance / object distance L between 
an object / object and the image pickup unit / lens [see Kojima, Figs. 1 09, 110 and 1 1 3, 
col. 30, lines 37-39], wherein the face image extraction part / detection of human face 
area includes a part for specifying a size of the face image / detect width and size of a 
human face to be extracted, based on a result of recognition by the distance recognition 
part / based on the object distance L [see Kojima, Figs. 1 09, 110 and 1 1 3, col. 30, lines 
37-39, col. 42, lines 53-65]. Therefore it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify Center's device using 
Kojima's teachings by including the distance recognition part to Center's face image 
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extraction part for improving the identification of a human face area (see Kojima, col. 42, 
lines 50-51). Therefore, the current claims are still not in condition for allowance 
because they are still not patentably distinct over the prior art references. 

Claim Objections 

7. Claims 1,13, and 23-26 are objected to because of the following informalities: 
Claims 1 and 23 , line 21 , claim 13 , line 31 , claim 24 , line 23, claim 25 , line 1 9, claim 26 , 
line 22 respectively: "a distance between an object and the image pickup unit" should be 
-- a distance between the object and the image pickup unit -. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 1 -3, 5-1 2, 23, and 25 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Center (US 2002/01 13862 A1 , as applied in previous Office Acton) in 
view of Kojima et al (US 5,638,136). 

Re Claim 1 : Center discloses an image pickup device / videoconferencing system (see 
paragraph [0002], [0027], [0018], abstract) comprising an image pickup unit / camera 
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including a lens / lens of camera and an image sensor / camera sensors such as zoom 
and focus sensors, and a control unit / computer system connected to camera for 
processing an image picked up by the image pickup unit and storing a processed image 
in one of an internal memory / memory of camera and computer system RAM or a 
predetermined storage medium (see Fig. 1, paragraphs [0018], [0003], and [0027], the 
computer is connected to the camera and they interact in real time since this is video 
conferencing), the control unit comprising a face image extraction part / locate face for 
extracting a face image contained in the image picked up by the image pickup unit (see 
Fig. 1, paragraphs [0018] and [0019]); an inference part / evaluate image quality for 
executing a process of inferring / evaluating attributes / information or characteristics of 
a person constituting an object based on feature amounts / color or motion analysis in 
an image area including the face image extracted (see paragraphs [0020], [0025], and 
[0004]); a parameter selecting part for selecting a set of parameters / skin color tones or 
templates from a table based on the attributes / information or characteristics or range 
of appearance of a person inferred by the inference part, wherein the table stores a 
plurality of sets of parameters that are respectively optimized based on each attribute of 
a person {see paragraphs [0028] and [0047], Center teaches (for color analysis) a 
lookup table which is trained [optimal over the default table] for a particular person 
[regular user] having the parameters based on attributes of a person [skin color tones 
for the range of the persons appearance], Center also teaches (for template matching 
method) selecting the templates [parameters] from a number of templates [table] 
[optimal because Center tries to achieve the greater detection accuracy] tuned to 
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recognize a particular person using the range of appearance or a particular user 
[parameters based on attributes of a particular person]}; an image pickup conditions 
adjusting part / adjust and control the camera for adjusting image pickup conditions / 
brightness, contrast and color balance based on the selected parameters by the 
parameter selecting part (see Fig. 1, paragraph [0020], [0028], [0047]); an information 
processing part / processing pass for storing in selected one of the memory and the 
storage medium / computer system RAM the image obtained / current frame or adjusted 
frame under the image pickup conditions adjusted by the image pickup conditions 
adjusting part (see paragraphs [0020], [0026], and [0027], the computer stores a current 
frame and does the evaluation of this information and then uses and produces an 
adjustment based on the evaluation and stores this adjustment frame). 

Although Center is silent in teaching the storing of the selected parameters in 
one of an internal memory and a predetermined storage medium, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made because 
the selected templates need to be stored in the internal memory in order to be accessed 
and used later on for Centers face recognition tracking system. 

However Center fails to disclose or fairly suggest a distance recognition part. 

Kojima discloses a distance recognition part / calculate object distance L for 
recognizing a distance / object distance L between an object / object and the image 
pickup unit / lens [see Kojima, Figs. 1 09, 1 1 0 and 1 1 3, col. 30, lines 37-39], wherein the 
face image extraction part / detection of human face area includes a part for specifying 
a size of the face image / detect width and size of a human face to be extracted, based 
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on a result of recognition by the distance recognition part / based on the object distance 
L [see Kojima, Figs. 109, 110 and 113, col. 30, lines 37-39, col. 42, lines 53-65]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify Center's device using Kojima's teachings by including 
the distance recognition part to Center's face image extraction part for improving the 
identification of a human face area (see Kojima, col. 42, lines 50-51). 

Re Claim 2: Center further discloses an inference part / evaluate image quality for 
executing the inference / evaluation of at least one of the race, age and sex as the 
attributes / range of appearance (see [0028], lines 4-9, [0048], the evaluation includes 
any combination of the template matching, motion detection, background differencing, 
and color analysis, in this case the template matching searches for a face using 
templates that represent the range of appearance of the different types of faces which 
are known as eigenfaces and these eigenfaces typically use race and sex). 

Re Claim 3: Center further discloses wherein the information processing part / 
processing pass includes a part for producing a link information / hypothesis containing 
a position / location of the face image extracted by the face image extraction part / 
locate face and the inference information / evaluation information obtained by the 
inference process / evaluation executed by the inference part, and wherein the link 
information / hypothesis is stored in selected one of the memory and the storage 
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medium / computer system RAM together with the image / current frame picked up by 
the image pickup unit (see paragraph [0048], lines 5-9). 

Re Claim 5: Center further discloses wherein the control unit / computer system 
includes a focal length adjusting part for adjusting the focal length of a lens of the image 
pickup unit / camera in accordance with the result of extraction by the face image 
extraction part / locate face (see paragraph [0004], lines 5-8, once the face is extracted 
and detected and the computer system adjusts the camera to automatically focus based 
on the detection and evaluation results). 

Re Claim 6: Center further discloses comprising a first operating unit / results Pyramid 
of visual detector for designating a range / location and size of extracting a face image, 
wherein the face image extraction part / locate face includes a part for limiting the face 
image extraction area in the image picked up by the image pickup unit / camera in 
accordance with the designating operation of the first operating unit (see paragraphs 
[0050] and [0004], with the range specified by the visual detector the camera is adjusted 
by focusing and zooming to the correct location and size of the face). 

Re Claim 7: Center further discloses comprising a first operating unit / color analysis for 
designating a deletion / remove and substitute of the result / skin probability of 
extracting a predetermined part / area of skin of the face image extracted, wherein the 
face image extraction part / locate face includes a part for updating the result of 
extracting the face image in accordance with the designating operation of the first 
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operating unit (see paragraphs [0004], [0046], and [0047], an update of the face area 
parameters of the color analysis is accomplished by the deletion of skin probability in 
the lookup table followed by the addition or substitution from the user and it is these 
parameters which update and adjust the brightness, contrast and color balance of the 
image). 

Re Claim 8: Center further discloses comprising a first operating unit for performing an 
operation of correcting / adjusting the inference information / information or 
characteristics of a person obtained by the inference process of the inference part / 
evaluate image quality, wherein the information processing part / processing pass 
includes a part for correcting the inference information in accordance with the correcting 
operation of the third operating unit (see paragraphs [0026] and [0020], abstract, the 
processing passes which are the template matching, motion detection, background 
differencing, and color analysis adjust the information of the face, paragraphs [0046] 
and [0047], the color analysis pass looks up and adjusts the skin area table values U 
and V for proper probability calculations, the U and V values are related to the color 
information as seen in [0027] lines 17-19). 

Re Claim 9: Center further discloses comprising a first operating unit / camera 
parameter unit for correcting the image pickup conditions / brightness, contrast and 
color balance adjusted by the image pickup conditions adjusting part / adjust and control 
the camera, wherein the image pickup conditions adjusting part includes a part for 
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readjusting the image pickup conditions in accordance with the correcting operation of 
the first operating unit (see Fig. 1 , paragraph [0004], lines 5-13, paragraph [0020]). 

Re Claim 10: Center further discloses the information processing part / processing pass 
includes a part for determining a direction / location of the face of an object in the image 
based on the result of extraction of the image stored in selected one of the memory and 
the storage medium by the face image extraction part / locate face, and a part for 
rotating / tilt movable camera the image in such a manner that the face direction 
conforms with a predetermined reference direction in the case where the determined 
face direction is different from the reference direction (see [0024], the wide-angle 
movable camera may tilt). 

Although Center doesn't specifically recite the limitation of determining the 
direction of the face and conforming the determined direction with the reference 
direction, it would be obvious to one of ordinary skill in the art at the time the invention 
was made to include this limitation in the processing pass in order to actually have 
Center's wide-angle movable camera tilt in the appropriate direction to actually optimize 
the face location detector (Center, paragraph [0024]). 

Re Claim 1 1 : Center further discloses comprising a feature amount / skin color 
probability storage part for storing the feature amount of the face image already 
extracted, wherein the face image extraction part includes a specified image extraction 
part / locate face for extracting an image area including the feature amount of the 
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specified face image stored in the feature amount storage part from the image picked 
up by the image pickup unit (see paragraph [0046], lines 22-27, a skin color probability 
is stored in colorPyramid after the face is extracted from the current incoming image 
and after it goes through the color analysis processing pass). 

Re Claim 12: Center further discloses comprising an object storage part / computer 
system RAM for storing the feature amount / skin tone color probability of the face 
image of a specified object / user or users, wherein the information processing part / 
processing pass compares / evaluates for modification the feature amount / skin tone 
color probability of the face image extracted by the face image extraction part / locate 
face with the feature amount / skin color probability stored in the object storage part / 
computer system RAM, so that in the case where the comparing process shows that the 
extracted face image is that of the specified object, a link information / hypothesis 
containing the inference information / information or characteristics of a person obtained 
by the inference process of the inference part / evaluate image quality and the 
information / U and V values for identifying the specified object / current face image is 
produced and stored in selected one of the memory and the storage medium / computer 
system RAM together with the image picked up / current image by the image pickup unit 
(see Fig. 1, [0046], lines 22-27, [0047], lines 1-5, [0048], lines 5-9, [0050]). 
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As to claims 23 and 25, the claims are the corresponding computer-readable 
medium and method claims to system claim 1 respectively. The discussions are 
addressed with regard to claim 1 . 

10. Claims 13-15, 17-22, 24 and 26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Center, in view of Soriano ("Making saturated facial images useful 
again", a Non-patent literature reference supplied by the Applicant, as applied in 
previous Office Action) and Kojima. 

Re Claim 13: Center discloses an image pickup device / videoconferencing system (see 
paragraph [0002], [0027], [0018], abstract) comprising an image pickup unit / camera 
including a lens / lens of a camera and an image sensor / camera sensors such as 
zoom and focus sensors; a control unit / computer system connected to the camera for 
processing an image picked up by the image pickup unit and storing processed image 
in selected one of an internal memory / memory of camera or computer system RAM 
and a predetermined storage medium (see Fig. 1, paragraphs [0018], [0003], and 
[0027], the computer is connected to the camera and they interact in real time since this 
is video conferencing), the control unit comprising a registration part / skin tone color 
probability lookup table for holding the feature amount / skin tone color probability of a 
face image of each of a predetermined number of objects / users (see [0047], lines 1-5, 
a predetermined number of objects are a multiple number of users) and information 
required for adjusting the optimum image pickup conditions / optimum brightness, 
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contrast and color balancing in correspondence with identification information / skin tone 
color unique to the object, a face image extraction part / locate face for extracting a face 
image contained in the image picked up by the image pickup unit (see Fig. 1 paragraphs 
[0018] and [0019]); an inference part / evaluate image quality for inferring / evaluating 
attributes / information or characteristics of a person by comparing the feature amount / 
color or motion analysis of the face image extracted by the face image extraction part 
with the information registered in the registration part (see paragraphs [0020], [0025], 
and [0004]); a parameter selecting part for selecting a set of parameters / skin color 
tones and templates from a table based on the attributes / information or characteristics 
or range of appearance of a person inferred by the inference part, wherein the table 
stores a plurality of sets of parameters that are respectively optimized based on each 
attribute of a person {see paragraphs [0028] and [0047], Center teaches (for color 
analysis) a lookup table which is trained [optimal over the default table] for a particular 
person [regular user] having the parameters based on attributes of a person [skin color 
tones for the range of the persons appearance], Center also teaches (for template 
matching method) selecting the templates [parameters] from a number of templates 
[table] [optimal because Center tries to achieve the greater detection accuracy] tuned to 
recognize a particular person using the range of appearance or a particular user 
[parameters based on attributes of a particular person]}; an image pickup conditions 
adjusting part / adjust and control the camera for adjusting the image pickup conditions / 
brightness, contrast and color balance for the image pickup unit based on the selected 
parameters by the parameter selecting part (see Fig. 1, paragraph [0020], [0028], 
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[0047]); an information processing part / processing pass for storing in selected one of 
the memory and the storage medium / computer system RAM the image obtained under 
the image pickup conditions adjusted by the image pickup conditions adjusting part (see 
paragraph [0020], [0026], and [0027], the computer stores a current frame and does the 
evaluation of this information and then uses and produces an adjustment based on the 
evaluation and stores this adjustment frame). 

Although Center is silent in teaching the storing of the selected parameters in the 
one of an internal memory and a predetermined storage medium, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made because 
the selected templates need to be stored in the internal memory in order to be accessed 
and used later on for Centers face recognition tracking system. 

However, Center fails to disclose or fairly suggest specifically that a registration 
part holds the information required for adjusting the optimum image pickup conditions in 
correspondence with identification information unique to the object. Also, Center fails to 
specifically suggest a distance recognition part. 

Soriano discloses information / ideal condition value required for adjusting the 
optimum image pickup conditions / brightness, contrast, and color balancing (Center 
discloses these pickup conditions) in correspondence with identification information / 
skin tone color (Center discloses this information) unique to the object (Soriano, Section 
2.2 Color Correction by RGB eigenfaces, Section 3.1 Physics-Based Face Database, 
abstract, lines 6-8, the pickup conditions are adjusted to the ideal condition values to 
recover color information in facial images taken under non-ideal conditions). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Center's device using Soriano's teachings by 
including the information required for adjusting the pickup conditions to Center's 
registration part in order to recover color information in facial images taken under non- 
ideal conditions (Soriano, abstract, lines 6-8). 

However Center, as modified by Soriano, still fails to specifically suggest a 
distance recognition part. 

Kojima discloses a distance recognition part / calculate object distance L for 
recognizing a distance / object distance L between an object / object and the image 
pickup unit / lens [see Kojima, Figs. 1 09, 1 1 0 and 1 1 3, col. 30, lines 37-39], wherein the 
face image extraction part / detection of human face area includes a part for specifying 
a size of the face image / detect width and size of a human face to be extracted, based 
on a result of recognition by the distance recognition part / based on the object distance 
L [see Kojima, Figs. 109, 110 and 113, col. 30, lines 37-39, col. 42, lines 53-65]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to further modify Center's device, as modified by Soriano, using 
Kojima's teachings by including the distance recognition part to Center's face image 
extraction part for improving the identification of a human face area (see Kojima, col. 42, 
lines 50-51). 

As to claim 15, the discussions are addressed with respect to claim 3. 
As to claim 17, the discussions are addressed with respect to claim 5. 
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As to claim 18, the discussions are addressed with respect to claim 6. 
As to claim 19, the discussions are addressed with respect to claim 7. 
As to claim 20, the discussions are addressed with respect to claim 8. 
As to claim 21 , the discussions are addressed with respect to claim 9. 
As to claim 22, the discussions are addressed with respect to claim 10. 

As to claims 24 and 26, the claims are the corresponding computer-readable 
medium and method claims to system claim 13 respectively. The discussions are 
addressed with regard to claim 13. 

Re Claim 14: Center further discloses the control unit / computer system includes a part 
/ automatic gain control (AGC) for receiving the input of the information required (taught 
by Soriano above) for adjusting the optimum image pickup conditions / optimum 
brightness, contrast, and color balancing and the identification information / skin tone 
color of the object in response to an image pickup operation / processing pass of a 
predetermined object for registration in the registration part / skin tone color probability 
lookup table, and storing / computer system RAM the input information in the 
registration part together with the face image of the object (see [0003]-[0004], [0046]- 
[0048], and [0050]). 

Although Center's AGC is part of the camera and not part of the control unit or 
computer system, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to have an AGC type component built into a computer and 
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controlled by the computer in order for the color correction or adjustment to meet its 
highest quality by having the automatic gain control type component be dynamically 
adjusted by the computers inputs. 

Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Terashita et al discloses methods of collecting photometric 
image data and determining light exposure by extracting feature image data from an 
original image. 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Bernard Krasnic whose telephone number is (571) 270- 
1357. The examiner can normally be reached on Mon-Thur 8:00am-4:00pm and every 
other Friday 8:00am-3:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
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you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



Bernard Krasnic 
April 15, 2008 
/Jingge Wu/ 

Supervisory Patent Examiner, Art Unit 2624 



